Papers with Natural language inference
A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus (2024.acl-long)
Copied to clipboard
| Challenge: | Natural language inference (NLI) is an actively studied topic serving as a proxy for natural language understanding. |
| Approach: | They propose to use a Romanian NLI corpus to analyze sentence pairs . they use multiple machine learning methods to establish competitive baselines . |
| Outcome: | The proposed model improves on the best model by employing a new curriculum learning strategy based on data cartography. |
KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing benchmark datasets for natural language inference and semantic textual similarity (STS) are not available in the Korean language. |
| Approach: | They construct and release new datasets for Korean NLI and STS . they machine-translate existing English training sets and manually translate development and test sets into Korean to accelerate research on Korean NLU. |
| Outcome: | The proposed datasets are available at https://github.com/kakaobrain/KorNLUDatasets. |
PhoBERT: Pre-trained language models for Vietnamese (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Experimental results show that PhoBERT outperforms the recent best pre-trained multilingual model XLM-R in multiple Vietnamese-specific NLP tasks. |
| Approach: | They present PhoBERT with two versions, Phobert-base and PhoBRET-large, which are pre-trained for Vietnamese. |
| Outcome: | The proposed model outperforms the best pre-trained model XLM-R and improves the state-of-the-art in multiple Vietnamese-specific NLP tasks including Part-of speech tagging, Dependency parsing, Named-entity recognition and Natural language inference. |
Stress Test Evaluation for Natural Language Inference (C18-1)
Copied to clipboard
| Challenge: | Existing models perform well at standard datasets for NLI, achieving impressive results across different genres of text. |
| Approach: | They propose to use automatic stress tests to evaluate models' ability to make inferential decisions. |
| Outcome: | The proposed model performs well across genres of text, but lacks the ability to make inferential decisions. |
Simple but Challenging: Natural Language Inference Models Fail on Simple Sentences (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Natural language inference (NLI) tasks are difficult to perform on large datasets . a small number of simple sentences can improve model performance, authors say . |
| Approach: | They propose to use syntactically simple sentences to test the inference ability of NLI models. |
| Outcome: | The proposed set of simple sentences shows that the models fine-tuned on MNLI and SNLI perform poorly on Simple Pair. |
Asynchronous Deep Interaction Network for Natural Language Inference (D19-1)
Copied to clipboard
| Challenge: | Existing methods have framed the reasoning problem as a semantic matching task. |
| Approach: | They propose an asynchronous deep interaction network (ADIN) to deconstruct the reasoning process and implement asynchron and multi-step reasoning. |
| Outcome: | The proposed model outperforms strong baselines on three popular benchmarks: SNLI, MultiNLI, and SciTail. |
“I’m Not Mad”: Commonsense Implications of Negation and Contradiction (2021.naacl-main)
Copied to clipboard
| Challenge: | a new commonsense knowledge graph for negated and contradicted events is developed to help humans reason about their underlying causes and effects. |
| Approach: | They propose a new commonsense knowledge graph with 624K if-then rules focusing on negated and contradictory events. |
| Outcome: | The proposed model can be used to analyze negated and contradicted statements in natural language. |
LawngNLI: A Long-Premise Benchmark for In-Domain Generalization from Short to Long Contexts and for Implication-Based Retrieval (2022.findings-emnlp)
Copied to clipboard
| Challenge: | a new benchmark for natural language inference is proposed for long premises . lawngNLI can train and test systems for implication-based case retrieval and argumentation. |
| Approach: | They propose a new natural language inference benchmark LawngNLI from U.S. legal opinions with automatic labels with high human-validated accuracy. |
| Outcome: | The proposed benchmark can train and test systems for implication-based case retrieval and argumentation. |
IMPLI: Investigating NLI Models’ Performance on Figurative Language (2022.acl-long)
Copied to clipboard
| Challenge: | Understanding figurative language is a difficult area in NLP but is essential for proper understanding. |
| Approach: | They propose to use a dataset to generate 24k semiautomatic pairs and manually create 1.8k gold pairs to evaluate NLI models. |
| Outcome: | The proposed models can detect entailment relationship between figurative phrases and their literal counterparts, but perform poorly on similar structured examples. |
BioNLI: Generating a Biomedical NLI Dataset Using Lexico-semantic Constraints for Adversarial Examples (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Biomedical research has progressed at a tremendous pace, with PubMed2 indexing well over 1M publications per year in the past eight years. |
| Approach: | They propose a semi-supervised procedure that bootstraps biomedical NLI datasets from positive entailment examples present in biomedically published texts. |
| Outcome: | The proposed procedure bootstraps biomedical NLI datasets from positive entailment examples from biomedically challenging texts. |
Identifying inherent disagreement in natural language inference (2021.naacl-main)
Copied to clipboard
| Challenge: | Natural language inference is the task of determining whether text is entailed, contradicted or unrelated to another piece of text. |
| Approach: | They propose to tease systematic inferences from disagreement items by capturing modes in annotations to simulate uncertainty in the annotation process. |
| Outcome: | The proposed approach performs statistically better than baselines on the CommitmentBank corpus in English. |
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference (2021.findings-acl)
Copied to clipboard
| Challenge: | Existing studies focus on sentence-level inference, which limits its application in downstream NLP problems. |
| Approach: | They propose to construct a large-scale dataset for document-level NLI that can be used to study NLP problems. |
| Outcome: | The proposed model performs well on popular sentence-level benchmarks and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity. |
InferLite: Simple Universal Sentence Representations from Natural Language Inference Data (D18-1)
Copied to clipboard
| Challenge: | InferLite is a lightweight version of InferSent that does not use recurrent layers and can generalize to multiple pre-trained word embeddings. |
| Approach: | They propose a lightweight version of InferSent that does not use recurrent layers and operates on a collection of pre-trained word embeddings. |
| Outcome: | The proposed model outperforms existing models that learn generic embeddings in an unsupervised setting, often requiring several days or weeks to train. |
Evaluating BERT for natural language inference: A case study on the CommitmentBank (D19-1)
Copied to clipboard
| Challenge: | Natural language inference datasets can identify premise-hypothesis relationship without observing premise . recasting of the CommitmentBank for NLI creates hypotheses that stand in entailment/contradiction/neutral relationship with premise. |
| Approach: | They propose to recast the CommitmentBank for NLI to stand in certain relationships with the premise . hypotheses are complements of clause-embedding verbs in each premise, rethinking the CommittedBank . |
| Outcome: | The proposed model performs well on the CommitmentBank with 85% F1 . however, the model does not capture the full complexity of pragmatic reasoning, authors say . |
New Protocols and Negative Results for Textual Entailment Data Collection (2020.emnlp-main)
Copied to clipboard
| Challenge: | Natural language inference data has proven useful in benchmarking and as pretraining data for tasks requiring language understanding. |
| Approach: | They propose four alternative protocols to improve annotation quality and diversity . they use 8.5k-example training sets to compare different protocols . |
| Outcome: | The proposed protocols improve the ease of training and quality of the examples. |
Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition (2020.acl-main)
Copied to clipboard
| Challenge: | Natural language inference (NLI) is an increasingly important task for natural language understanding . however, the ability of NLI models to make pragmatic inferences remains understudied . |
| Approach: | They use semi-automatically generated sentence pairs to evaluate whether NLI models make pragmatic inferences. |
| Outcome: | The proposed model trains on multiNLI and shows that it learns to draw pragmatic inferences. |
Improving the robustness of NLI models with minimax training (2023.acl-long)
Copied to clipboard
| Challenge: | Experimental results show that our method consistently outperforms other robustness enhancement techniques on out-of-distribution adversarial test sets, while maintaining high in-distance accuracy. |
| Approach: | They propose a minimax objective between a learner model being trained for the task and an auxiliary model aiming to maximize the learner's loss by up-weighting underrepresented "hard" examples with patterns that contradict the shortcuts learned from the prevailing "easy" examples. |
| Outcome: | The proposed method outperforms other robustness enhancement techniques on out-of-distribution adversarial test sets while maintaining high in-distance accuracy. |
SI-NLI: A Slovene Natural Language Inference Dataset and Its Evaluation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing datasets for natural language inference (NLI) are limited to English and a few other well-resourced languages. |
| Approach: | They propose to use a dataset for natural language inference to extend the resources for the task. |
| Outcome: | The proposed dataset is constructed from scratch using knowledgeable annotators with carefully crafted guidelines aiming to avoid common problems in existing datasets. |